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Abstract — Neural Networks have been under examination 
for decades in many areas of research, with varying degrees 
of success and acceptance. Key goals of computer learning, 
rapid problem solution, and automatic adaptation have been 
elusive at best. This paper 1,2 summarizes efforts at NASA’s 
Marshall Space Flight Center harnessing such technology to 
autonomous space vehicle docking for the purpose of 
evaluating applicability to future missions. 
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1. Introduction 

One research laboratory at Marshall Space Flight Center 
(MSFC) is the Flight Robotics Lab. There, a particularly 
impressive piece of infrastructure is the Flat Floor, a large 
precision-poured epoxy surface designed to be planar to 
within about seven thousandths of an inch over large areas 
at a time. This allows levitation of very heavy test “sleds” 
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on the floor using air bearings fed from high pressure 
sources, these typically being contained on the test vehicle 
itself. Given such resources, nearly ffictionless conditions 
can be produced for study of the motion of objects in three 
degrees of freedom. A prime use is found testing schemes 
for autonomous rendezvous and docking (AR&D) in Space 
[1], Figure 1 shows the Small Air Sled (SAS) on the Flat 
Floor. 

Over the past few years, personnel at MSFC have been 
pursuing access to patented (e.g.: [3] through [10]) artificial 
neural network (ANN) technology developed by 
Imagination Engines, Inc. (IEI). A number of appealing 
target applications readily present themselves; however, 
AR&D is perhaps the most appealing as a representation of 
NASA’s core missions. 



Figure 1: SAS and Target on the MSFC Flat Floor 



This work was made possible through the Independent 
Research and Development (IR&D) program at MSFC [2], 
Objectives of the research include demonstration of ANNs 
in performance of various AR&D functions on the MSFC 
Flat Floor utilizing video camera inputs as a primary data 
source, realizing the networks in both software and 
hardware forms. 

2. Background and Theory 

IEI’s ANN technologies designated “Self-Training 
Autonomous Neural Network Objects” (STANNOs) and 
“Creativity Machines” have already been developed to a 
high level of capability through many years of 
experimentation. Implementation for specific tasks is often 
accomplished in minutes or hours, building upon a set of 
tools and experience previously groomed and tested in 
various other operations. 

One major strength of STANNOs is ability to build, in 
software form, extremely large, multilayer ANNs capable of 
processing millions of bytes streaming from relatively high- 
resolution video sources (640 x 480). A modestly appointed 
computer is thereby capable of completing both forward- 
and back-propagation steps on millisecond time scales. The 
ability to train and execute multi-billion-weight ANNs on 
digital computers has enabled nearly instantaneous 
translation of input camera frames from a robot — for 
example, its forward view — to navigational fields for the 
robot to follow toward pre-designated targets. 


application of stimuli to the network, new and often 
remarkably unique results are output for consideration. 

Figure 2 shows a diagram of one adaptation of STANNO 
technology. In this configuration, the feedback topology 
mentioned above is shown as part of the ANN module. The 
figure serves to illustrate core means by which these ANNs 
train themselves: human interactions in effect involve only 
instruction of the ANNs in their assigned objectives, relying 
upon the ANNs to perform the more detailed and intensive 
neural training much faster than possible through 
conventional means. The final trained configurations are 
derived by experimentation, evaluation of results, and 
updated programming of neural weights. 
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Figure 2: System Conceptual Diagram 


Creativity Machines are exemplified by ANN pairs. One 
side is pre-trained from sample data patterns embodying a 
targeted conceptual space. As it is internally or externally 
perturbed stochastically or systematically, this net generates 
output patterns representing potentially novel concepts or 
plans of action. The other side serves as a critic, selecting 
those ideas or strategies meeting criteria implicitly instilled 
within it via supervised learning. In the case of a control 
system, the perturbed net generates control strategies that 
are implemented upon approval by the critic net. 

If a Creativity Machine is STANNO-based, it may 
cumulatively learn from its own mistakes, bootstrapping 
from no learning whatsoever, to progressively higher levels 
of competence. 

Furthermore, if the critic is allowed to correct connection 
weights within the former net, it may react to inaccuracies 
therein by iteratively correcting connection weight values. 
In this way, the Creativity Machine concept leads directly to 
that of STANNOs. 

The Creativity Machine itself absorbs knowledge of the 
problem space under consideration and then explores 
“confabulations” of the information to arrive at new 
possibilities for solutions. Through carefully controlled 


As most people familiar with basic ANN theory know, 
when network weights are locked and cannot be further 
changed, the network is considered “static.” Allowing 
weights to continue training during operation categorizes the 
network as “dynamic.” 

A very exciting aspect of this technology is that the ANNs 
not only learn behavior that can be creatively manipulated, 
but also learn constraints to be applied in an operational 
system. This contrasts very favorably with other 
techniques, including other ANN technologies and Genetic 
Algorithms, which require much fuller knowledge of the 
problem space’s constraints prior to training or 
programming the intelligent core. 

A very important possibility opens up to experimenters in 
view of this and the modular nature of STANNO 
technology. Consider making the Constraints Network 
static while allowing the Creativity Network to continue in 
dynamic operation. This scheme retains the benefits of 
dynamic network adaptation while keeping responses under 
fixed constraints. In fact, the Constraints Network can then 
potentially have standardized Independent Verification and 
Validation (IV&V) applied in order to facilitate man-rating 
or other qualification. 
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3. Early Results 

In a set of impromptu experiments carried out on MSFC’s 
Flat Floor during a preliminary visit in May 2006, IEI 
demonstrated basic capabilities of this technology in 
application to autonomous docking. The machine vision 
system utilized a Logitech webcam plugged into a laptop 
and taped to the top of the air sled. The image recognition 
package, developed for an entirely different purpose, 
consisted of a cascade of autonomously interconnecting 
STANNOs. This self-connecting ANN cascade was trained 
on four discreet images of the docking target from four 
evenly-spaced standoff ranges. Further, each of these 
images was approximately centered on the vertical midline 
of the target. 

Video footage resulting from this exercise, illustrated in 
Figure 3, depicts the ANN’s capability to not only identify 
cases in which white crosshairs generated in the images are 
nearly centered on the target, but also identify which range 
of distances to the target is most closely represented. Thus, 
the very rudimentary experiment shows how we may expect 
to estimate both alignment with and range to the target. The 
ability to form a crude navigational field useful for guiding 
the SAS is a result. 
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Figure 3: Early Impromptu Flat Floor Experimentation 

It should be noted this demonstration was accomplished in 
less than an hour of actual time on the Flat Floor, and less 
than one minute total training of the ANN system to 
recognize the four images. Furthermore, we emphasize that 
the work was done without stereo imagery, with a 
commercial webcam purchased for less than one hundred 
dollars. 

4. First Series of Tests 

In early September 2006, IEI returned to MSEC for more 
testing on the Flat Floor. The first several days were spent 
making connection of the IEI laptop to the SAS control 
circuitry. Cabling was introduced to bypass the computer 


ordinarily utilized to command air thrusters on and off and 
maneuver the SAS. 

The remainder of this visit was spent implementing schemes 
to adapt outputs from a self-connecting ANN cascade to the 
SAS. These ANNs generated a navigational field that 
would tend to rotate the SAS clockwise or counterclockwise 
in order to align with the target. In this case, thruster 
commands were coded directly in LabVIEW on the IEI 
laptop, in order to give researchers the chance to become 
more comfortable with the behavior of the air sled. Such 
caution seemed prudent because of restrictions on rates of 
SAS motion and general unfamiliarity of the researchers 
with the hardware. The ANNs were further trained to 
produce an attractive navigational field and command light 
forward thrusting in the event the SAS was aligned with the 
target. Finally, they were trained to produce a repulsive 
navigational field, resulting in light backward thrusting, to 
achieve station keeping just short of the target. Note that 
hard dockings were generally not a goal in this series of 
experiments. 

After reasonable assurance of basic control and safety, a 
number of sequential runs with the SAS showed the 
STANNO cascade capable of accomplishing rendezvous 
with the target and initial action to take up station keeping at 
a range of about 3 meters. At this point it was discovered 
that because of the positions of thrusters selected for braking 
— in this instance at the apex of the SAS triangle - along 
with natural mismatching of individual thrust powers, 
braking caused the SAS to veer off-axis and break contact 
of the camera system with the target. Because searching for 
the target when out of view was not yet implemented, this 
generally terminated test runs. 



Figure 4: Front Panel Display during First Tests 


Nevertheless, in several sequential tests the SAS came to 
satisfactory alignment and was under control at the station 
keeping point prior to brake firing, and in at least one case 
the system managed to recover and all but fully dock with 
the target; because no braking was implemented inside of 
the station keeping point, hard docking had to be prevented 
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manually. Figure 4 shows the laptop display during this 
run. Note the small white crosshairs to either side of the 
central object in the video frame in the upper left; these 
mark the centers of interest of the ANNs, which served to 
cooperate in driving the navigational potential toward the 
docking fixture. 

5. Second Series of Tests 

The next logical step was to give the STANNO cascade 
more direct control over SAS air thrusters, rather than 
coding their timing and duration patterns directly. 
Ultimately, the ANNs should be able to fire random thrust 
patterns, observe results, and rapidly develop control 
schemes necessary to dock with the target routinely and 
efficiently. 


Incremental advancements would be made. Two different 
activities were undertaken to set the stage for 
accomplishment of these goals. 



Figure 5: STANNO-based Navigation Field Generation 


First, a video webcam was mounted and aligned to the front 
of the sled, as can be seen on close examination of Figure 1. 
Several frames of the target in different light conditions and 
from different perspectives were then presented to a 
STANNO cascade designed to now generate a navigational 
field or “channel” toward the docking target. Through prior 


offline training, this cascade became capable of converting, 
in real time, camera frames from the Logitech camera to a 
trough-like navigational field. This channel had a repulsive 
field that tended to restore the sled to the center of the 
trough, while constantly orienting it to face the target. 

Next, a LabVIEW routine was developed to apply random 
thrusts on the SAS while collecting data on the its position 
and orientation within the STANNO-generated navigational 
field, as well as supplementary range, speed, and angular 
information gleaned directly from complementary sensor 
data. 

These tests were delayed until December 2006, at which 
time this paper was overdue for publication. In the limited 
time available, it was only possible to demonstrate a 
reasonable semblance of ability to maintain forward motion 
of the sled within the navigation channel. However, 
because it was possible to produce only a small number of 
training exemplars for the neuro-control system, the sled 
tended to escape the navigational field. 

Figure 5 through Figure 7 illustrate key results from these 
tests. The navigation channel is defined by blue level 
indicators to the right of each successive frame. Note that 
the actual point of interest on the target is no longer the 
very center of the black cross member, but in these tests 
is an optical target near the middle of the arm to the left. 



Figure 6: SAS Outside the Navigation Channel 
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In Figure 5, as the sled travels down the center of the 
channel, the center blue indicator is lower than those to 
either side, as designed. Should the sled stray from this 
channel, the indicator pattern becomes asymmetric, as in 
Figure 6. 



Figure 7: Channel Invariance with Illumination Change 


Figure 7 illustrates the STANNO-generated navigation 
channel’s relative immunity to variations in illumination. 
As ambient lighting is changed, the machine vision system 
correctly indicates the SAS is centered on the channel. In 
any of these lighting environments, deviation from the 
channel also produces appropriate asymmetric, repulsive 
fields consistent with the responses in Figure 6. 

6. Plans for Continued Work 

In coming months, the control system’s training will be 
supplemented to better confine the SAS to the navigation 
channel. 

Initially, this will mean generation of more and better 
training samples, along with some refinement of the ANN s’ 
implementations, in order to more fully complete the 
objectives of the second set of tests. 


The next step in planned work is to give STANNOs 
complete control over the air thrusters, with certain 
constraints in place for safety and efficacy. This will allow 
the ANNs to demonstrate ability to experimentally learn air 
sled operation and then autonomously control docking. In 
addition, it will allow the ANNs to learn multiple 
capabilities, including how to compensate for differences in 
thrust force from one air thruster to another, thrust pressures 
changing with supply and demand, unequal placement of 
thrusters, and imbalances in SAS load. Additionally, they 
should learn how to accomplish reasonably smooth and 
efficient motion overall. In fact, given enough time for the 
research, it would be productive to operate the ANNs 
dynamically and allow them to compensate for simulated 
thruster failures, shifts in sled load, and other changes in 
SAS handling characteristics. 

Capacity for estimation of range will be calibrated against 
existing position-sensing instrumentation on the SAS, and 
then the ANNs will independently carry out a series of 
exercises designed to replicate standard Flat Floor docking 
work, but with only the benefit of video inputs. 

First, the STANNO-controlled sled will carry out 
autonomous docking from an initial attitude with the 
docking target already in view and the air sled orthogonally 
aligned with the target. 

Subsequent experiments will build upon these results, 
yielding capabilities not typically encountered in Flat Floor 
operations. 

Continuing exercises will progressively show the ability of 
ANNs to successfully achieve autonomous dockings from 
non-orthogonal initial alignments and alignments which 
start without the target initially in view, or even with the 
target obscured by intervening objects. 

To date, STANNOs have been realized primarily in 
software form, the most recent of which executes on a dual- 
processor laptop computer. If the target system is too small 
to house and power the laptop, as is often the case with 
small robotic platforms in particular, interfaces are made by 
external umbilical or wireless connection. 

This IR&D projects as one product the realization of 
STANNOs and their adaptations in hardware form. At least 
initially, that form will most likely be grids of 
interconnected Field Programmable Gate Array (FPGA) 
devices. These grids are commonly called an “FPGA 
Fabric,” as they knit together any number of FPGA devices, 
often along with other supporting electronics such as 
microprocessors, memory, and communication processors. 
Development of this capability is already underway. 

Ultimately, it is expected that ANN processing will be done 
completely in hardware, making it possible not only to 
realize exceptionally dramatic increases in operational 
speeds, but also to implement more and larger networks. 
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Eventual realization in ASIC or other forms is already under [10] Thaler, S. L. (2000), US Patent 6,356,884 . Device for the 
consideration. autonomous generation of useful information, 3/12/2002. 


7. Conclusions 


[11] Thaler, S. L. (1996) "Neural Networks that Create and 
Discover". PC AI , May / June, 1996 


Cumulatively over the course of this investigation, it has 
been shown that cascades of ANNs may be trained to 
generate navigational fields from raw camera frames. These 
ANN-generated navigational fields will potentially serve as 
virtual ‘fly-through’ channels to facilitate AR&D with space 
vehicles. 

Preliminary data indicate that additional ANN cascades can 
learn to produce thrust sequences for aligning spacecraft 
with and propelling them through such navigational 
channels. Additional time at the Flat Floor or satisfactorily 
equivalent facilities will be required to train underlying 
ANNs and conduct more complete demonstrations. 
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